Robotic control device and method for manipulating a hand-held tool

ABSTRACT

Described is a robotic control device for manipulating a gripper-held tool. The device includes a robotic gripper having a plurality of tactile sensors. Each sensor generates tactile sensory data upon grasping an tool based on the interface between the tool and the corresponding tactile sensor. In operation, the device causes the gripper to grasp a tool and move the tool into contact with a surface. A control command is used to cause the gripper to perform a pseudo-random movement with the tool against the surface to generate tactile sensory data. A dimensionality reduction is performed on the tactile sensory data to generate a low-dimensional representation of the tactile sensory data, which is then associated with the control command to generate a sensory-motor mapping. A series of control commands can then be generated in a closed-loop based on the sensory-motor mapping to manipulate the tool against the surface.

FIELD OF INVENTION

The present invention relates to a robotic control device and, moreparticularly, to a system and method for robotic manipulation of ahand-held tool.

BACKGROUND OF INVENTION

The present invention is related to robotic manipulation of an item tosubstantially improve the dexterity of robotic manipulation in light ofuncertainty. Uncertainty is present in the hand-tool interface:currently, it is still close to impossible for a robotic hand to graspobjects at predetermined contact points and forces. Moreover,uncertainty is present in the tool-environment interface (e.g., betweena pen tip and paper) as friction is still hard to model and to predict.Prior methods that analytically model a grasp failed under suchuncertainty.

Previous efforts in the area of robotic manipulation under uncertaintyfocused on using low-gain compliant control. Such control avoids hardcollisions, but cannot provide precision control if required (e.g., forwriting). To improve precision, some efforts used learning methods tocompute control torques without increasing the control gains fortrajectory tracking. Through random exploration (motor babbling), therobot learns the kinematic and dynamic relationship between joint anglesor torques and hand position. These efforts were limited to learning thekinematics and dynamics of the robot arm itself and thus could not copewith an uncertain interface between robot gripper (e.g., hand) andmanipulated object.

Recent prior art has dealt with the uncertain interface between thegripper and object. For example, researchers Kemp and Edsinger found amethod to obtain the position of a tool tip without knowing the contactpoint between tool and gripper. They described their process at “Robotmanipulation of human tools: Autonomous detection and control of taskrelevant features,” 5th IEEE International Conference on Development andLearning, 2006. As described by Kemp and Edsinger, the robot determinesthe tip position by waving the gripper and computing the image locationof highest speed. While operable for determining the tip position, themethod is limited to certain tool shapes and requires visual feedbackvia a visual sensor (e.g., video camera).

As opposed to visual sensor, tactile sensors sense a contact sensation.However, existing tactile sensors are still very noisy and have not beenpreviously used to analyze the uncertain interface between the gripperand tool. So far, the utility of tactile sense in robots is largelyreduced to on/off switches. In contrast, humans greatly enhance theirmanual dexterity through tactile sense. Blind people demonstrate thatgreat dexterity is possible using only tactile feedback. In addition,fine motor skills in healthy humans are hampered if tactile sense isremoved (e.g., lighting a match with anaesthetized fingers is almostimpossible). Thus, if tactile feedback can be efficiently exploited,robotic manipulation will become more feasible in human-like settings.

In summary, over the last several decades, many research groups aroundthe world have worked on robotic manipulation, but the uncertainty of agrasp has prohibited dexterous tool use. Tactile sense has not been usedefficiently for manipulation. The robotic field focused either onpredicting sensory input analytically or triggering purely reactivebehavior given sensory input. The first is limited by the noise of thesensory input, and the second prohibits gradual change of forceapplication.

Thus, a continuing need exists to extend robotic control into thetactile domain by allowing a more gradual change of force applicationthat smoothly adapts to changes in the environment (e.g., surface slopefor writing).

SUMMARY OF INVENTION

The present invention relates to a system and method for roboticmanipulation of a hand-held tool. For example, the present inventionincludes a robotic control device for manipulating such a hand-heldtool. The device includes a gripper (e.g., robotic hand) that is mobilein at least one degree-of-freedom. A plurality of tactile sensors isattached with the gripper. Each sensor is operable for generatingtactile sensory data upon grasping a tool based on the interface betweenthe tool and the corresponding tactile sensor. Examples of such tactilesensory data include a magnitude and direction of a force input. Tocontrol the gripper and manage the tactile sensory data, a computer iscommunicatively connected with both the gripper and the tactile sensors.The computer includes both a memory and a data processor. The memory isspecifically encoded with instructions, that when executed, cause thedata processor to perform the operations listed herein.

In operation, the device causes the gripper to grasp a tool and move thetool into contact with a surface. A control command is used to cause thegripper to perform a movement (e.g., sinusoidal curve, pseudo-randommovement, etc.) with the tool against the surface to generate tactilesensory data. A dimensionality reduction is performed on the tactilesensory data to generate a low-dimensional representation of the tactilesensory data, which is then associated with the control command togenerate a sensory-motor mapping. A series of control commands can thenbe generated in a closed-loop based on the sensory-motor mapping tomanipulate the tool against the surface.

In another aspect, the present invention includes an image sensor forcapturing an image of the tool. In this aspect, the computer is furtherconfigured to cause the processor to perform operations of capturing animage of the tool; identifying a desired point on the tool for movinginto contact with the surface; and generating a series of controlcommands in a closed-loop to cause the gripper to manipulate the tooluntil the desired point is moved into contact with the surface.

Finally, the present invention also includes a computer program productand computer implemented method. The computer program product includescomputer-readable instructions stored on a non-transitorycomputer-readable medium that are executable by a computer having aprocessor for causing the processor to perform the listed operations.Alternatively, the method comprises an act of causing a computer havinga processor to execute instructions specifically encoded on a memory,such that upon execution, the processor performs the operations.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects, features and advantages of the present invention will beapparent from the following detailed descriptions of the various aspectsof the invention in conjunction with reference to the followingdrawings, where:

FIG. 1 is a block diagram depicting the components of a robotic controlsystem of the present invention:

FIG. 2 is an illustration of a computer program product embodying thepresent invention:

FIG. 3 is an illustration of a gripper (i.e., robotic hand) with tactilesensors and corresponding tool to be grasped:

FIG. 4A is a front-view illustration of the gripper and grasped tool:

FIG. 4B is a side-view illustration of the gripper and grasped tool; and

FIG. 5 is a process now diagram, depicting flow from moving the roboticgripper for exploration to learning a sensory-motor relationship, whichis then used in a closed loop to control the gripper;

FIG. 6 is a graph depicting a distribution of sensory input data duringexploration;

FIG. 7 is an image of two graphs that illustrate how a low-dimensionalrepresentation of sensory data is plotted against a corresponding robotaction;

FIG. 8 is an illustration of a gripper (i.e., robotic hand) with animage sensor and a corresponding tool to be grasped;

FIG. 9 is a graph illustrating a relationship between robotic gripperlocation and low-dimensional representation of tactile input; and

Appendix A is a paper by the inventors of the present application inwhich they further describe how the online learning of tactile feedbackallows adaptation to unknown hand-tool and tool-environment interfaces.The Appendix is hereby incorporated, in its entirety, by reference asthough incorporated herein and is to be considered an integral part ofthis specification.

DETAILED DESCRIPTION

The present invention relates to a robotic control device and, moreparticularly, to a system and method for robotic manipulation of agripper-held tool. The following description is presented to enable oneof ordinary skill in the art to make and use the invention and toincorporate it in the context of particular applications. Variousmodifications, as well as a variety of uses in different applicationswill be readily apparent to those skilled in the art, and the generalprinciples defined herein may be applied to a wide range of embodiments.Thus, the present invention is not intended to be limited to theembodiments presented, but is to be accorded the widest scope consistentwith the principles and novel features disclosed herein.

In the following detailed description, numerous specific details are setforth in order to provide a more thorough understanding of the presentinvention. However, it will be apparent to one skilled in the art thatthe present invention may be practiced without necessarily being limitedto these specific details. In other instances, well-known structures anddevices are shown in block diagram form, rather than in detail, in orderto avoid obscuring the present invention.

The reader's attention is directed to all papers and documents which arefiled concurrently with this specification and which are open to publicinspection with this specification, and the contents of all such papersand documents are incorporated herein by reference. All the featuresdisclosed in this specification. (including any accompanying claims,abstract, and drawings) may be replaced by alternative features servingthe same, equivalent or similar purpose, unless expressly statedotherwise. Thus, unless expressly stated otherwise, each featuredisclosed is one example only of a generic series of equivalent orsimilar features.

Furthermore, any element in a claim that does not explicitly state“means for” performing a specified function, or “step for” performing aspecific function, is not to be interpreted as a “means” or “step”clause as specified in 35 U.S.C. Section 112. Paragraph 6. Inparticular, the use of “step of” or “act of” in the claims herein is notintended to invoke the provisions of 35 U.S.C. 112, Paragraph 6.

Before describing the invention in detail, first a description ofvarious principal aspects of the embodiments of the present invention isprovided. Subsequently, an introduction provides the reader with ageneral understanding of the principles of the present invention. Next,details of the embodiments of the principles of the present inventionare provided to give an understanding of the specific aspects. Finally,a brief synopsis is provided of the present invention.

(1) Principal Aspects

The present invention has several embodiments and may include additionalembodiments than those described herein. The first is a robotic controlsystem. The robotic control system is typically in the form of acomputer system operating software or in the form of a “hard-coded”instruction set. This system may be incorporated into a wide variety ofdevices that provide different functionalities, including the hardwareof the robotic gripper (or hand) and the corresponding tactile sensors.The second principal aspect is a method, typically in the form ofsoftware, operated using a data processing system (computer). The thirdprincipal aspect is a computer program product. The computer programproduct generally represents computer-readable instructions stored on anon-transitory computer-readable medium such as an optical storagedevice, e.g., a compact disc (CD) or digital versatile disc (DVD), or amagnetic storage device such as a floppy disk or magnetic tape. Other,non-limiting examples of computer-readable media include hard disks,read-only memory (ROM), and flash-type memories. The term “instructions”generally indicates a set of operations to be performed on a computer,and may represent pieces of a whole program or individual, separable,software modules. Non-limiting examples of “instruction” includecomputer program code (source or object code) and “hard-coded”electronics (i.e. computer operations coded into a computer chip). The“instruction” may be stored in the memory of a computer or on acomputer-readable medium such as a floppy disk, a CD-ROM, and a flashdrive. These aspects will be described in more detail below.

A block diagram depicting the components of a robotic control system ofthe present invention is provided in FIG. 1. The system 100 comprises aninput 102 for receiving information from at least one sensor (e.g.,tactile sensor) for use in detecting a surface or an applicable part ofthe hand-held tool. Note that the input 102 may include multiple“ports.” Typically, input is received from at least one sensor,non-limiting examples of which include tactile sensors and video imagesensors. An output 104 is connected with the processor for providinginformation regarding the surface and/or presence and/or identity ofhand-held tool(s) in the scene to other systems in order that a networkof computer systems may serve as a robotic control system. Output mayalso be provided to other devices or other programs; e.g., to othersoftware modules, for use therein. The input 102 and the output 104 areboth coupled with a processor 106, which may be a general-purposecomputer processor or a specialized processor designed specifically foruse with the present invention. The processor 106 is coupled with amemory 108 to permit storage of data and software that are to bemanipulated by commands to the processor 106.

An illustrative diagram of a computer program product embodying thepresent invention is depicted in FIG. 2. The computer program product isdepicted as an optical disk 200 such as a CD or DVD, or as a floppy disk202. However, as mentioned previously, the computer program productgenerally represents computer-readable instructions stored on anycompatible computer-readable medium.

(2) Introduction

Robotic manufacturing in cooperation with humans and robotic assistancein a human environment require the handling of tools that a robot graspsin an imprecise way. The present invention provides a system tomanipulate a tool within a robot gripper without detailed knowledge ofthe gripper-tool interface. The robot gripper is equipped with tactilesensors, which feed information to a computer that controls the gripper.In particular, the present invention allows applying a controlledpressure with the tip of the tool (e.g., a pen) on a surface despiteuncertainty about how the tool is held in the gripper and uncertaintyabout the structure of the surface. A unique aspect of this invention isa process that (a) learns a mapping between tactile feedback and toolcontrol through random exploration and dimensionality reduction and (b)uses this mapping to control the robot gripper.

(3) Details of the Invention

A purpose of this invention is a process for controlled forceapplication with a tool in a robotic gripper. The invention addressesthe difficulty to hold a tool exactly in a pre-computed posture. Thus,this posture will be uncertain. To control the tool despite thisuncertainty, the robot explores the tactile feedback resulting from itsactions and learns an association between the two.

Elements of the present invention are depicted in FIG. 3. As shown, thepresent invention can be embodied as an apparatus that includes arobotic arm and gripper 300. It should be understood that the roboticgripper 300 operates as a gripper to grasp a hand-held tool (i.e.,object). Thus, in its most simple aspect, the robot gripper 300 can bereferred to interchangeably as a gripper that is mobile in at least onedegree of freedom. For example and as depicted in FIGS. 4A and 4B, asimple gripper 400 is used to grasp a tool 402 (e.g., pencil). Howeverand referring again to FIG. 3, the robotic gripper 300 is not limited toa simple gripper (as depicted in FIGS. 4A and 4B), but can also be amore complicated, multi-directionally mobile robotic hand (as depictedin FIG. 3) that is mobile in many degrees-of-freedom.

To allow the robotic gripper 300 to sense contact with the tool, aplurality of tactile sensors 302 are positioned on or otherwise attachedwith the robotic gripper 300. The tactile sensor 302 is any suitablesensor that is capable of generating tactile sensory data upon graspinga tool (e.g., sensing touch, force, or pressure) and that can beemployed at an interface between the robotic gripper 300 (via thecorresponding tactile sensor) and the tool. A non-limiting example of asuitable sensor is the Shadow Tactile Sensor as produced by the ShadowRobot Company. Ltd., 251 Liverpool Road, London, NI ILX, UnitedKingdomn. The tactile sensors 302 can be positioned at any suitablelocation about the gripper 300. As a non-limiting example, the gripper300 includes robot fingers, with the tactile sensors 302 positioned onthe robot fingers.

The invention also includes a computer 304 to process sensory data(through a sensory interface 306) and to control the robot. For thiscontrol, the computer 304 may be linked to an external micro-controller(control circuit 308) that provides the control commands to the robotichardware, e.g., torques at the arm joints, etc. In addition, an externalcircuit (sensory interface 306) may interface with the tactile sensors302. The robot gripper 300 and computer 304 are linked through wires ora wireless connection.

As can be appreciated by one skilled in the an, this specification isnot directed to low-level robot control, as such controllers arecommonly available. Instead, the present invention is directed to asystem that is able to learn to manipulate a hand-held tool using therobot gripper 300. As noted above, a problem with manipulating ahand-held tool is the uncertainty between the hand-tool interface. Tocontrol the tool despite this uncertainty, the principles of theinvention teaches use of the tactile feedback resulting from the robot'sactions and learns an association between the two.

Initially and as depicted in FIG. 5, the system must be engaged to causethe gripper (robot) to grasp 500 a tool and move the tool into contactwith a surface. The tool is also in contact with the robot gripper'stactile sensors. Though, it does not need to be in contact with allsensors. From this initial condition, the invention proceeds as follows.A control command is generated to cause the robot arm (or gripper) toperform a movement 502 with the robot gripper move the tool into contactwith the surface and generate tactile sensory data. The movement 502 isany motion that brings the tool sufficiently often in contact with thesurface, non-limiting examples of such movements include randommovements, zig-zag movements, rectangular oscillation, periodicmovements (e.g., sinusoidal curve), and pseudo-random movements. Themotion must be “sufficient” to generate enough tactile sensory data toperform a dimensionality reduction 504.

Alternatively, instead of a movement, the computer may control exertionof a force at the gripper. In either event, tactile sensory data isgenerated at the interface between the gripper and the tool. The tactilesensory data is any suitable data that is indicative of the tactileresponse between the tool and the gripper, non-limiting examples ofwhich include a magnitude of a force input and a direction of a forceinput. The computer stores the tactile sensory data during this movementor force application.

Thereafter and as alluded to above, the system performs a dimensionalityreduction 504 of the tactile sensory data to generate a low-dimensionalrepresentation of the tactile sensory data. Thus, the computer finds alow-dimensional representation of the sensory data. As can beappreciated by one skilled in the art, there are several suitabletechniques for generating a low-dimensional representation of thetactile sensory data, a non-limiting example of which includes, for alinear distribution, principal component analysis to find the directionof a maximum variance and then projection of the data onto the principalcomponent. This technique was described by K. I. Diamantaras and S. Y.Kung, in “Principal Component Neural Networks,” Hoboken, N.J.; Wiley,1996. Through the dimensionality reduction step, a lower-dimensionalrepresentation is obtained for the sensory state. The dimensionalityreduction eliminates the dependence of this state on the grasp postureof the robot. An example is depicted in FIG. 6, which is a graphdepicting a distribution of sensory input during exploration. Thesensory values (si) vary systematically depending on the motor command;i.e., they lie on a lower dimensional manifold (dashed curve). In thisnon-limiting example, only three sensory dimensions are shown forillustrative purposes. Note that a “manifold” is understood by thoseskilled in the art and as defined by Wikipedia as “a topological spacethat resembles Euclidean space near each point” (See Wikipedia.org,“Manifold”). Thus, a lower-dimensional manifold is a space embedded in ahigher dimensional space. In the field of machine learning,“lower-dimensional manifold” refers to the phenomenon that data areusually high dimensional, but locally constrained to fewer dimensions,and thus the data's distribution could be described by alower-dimensional manifold.

Next, the system learns a relationship (506) between the low-dimensionalrepresentation of the tactile sensory data and the control command forthe robot arm gripper. This learning involves first collectingcorresponding pairs of sensory data 704 and control commands. The rightgraph in FIG. 7 (702) shows data points in the joined space of sensorydata (X) and control commands (A). Second, learning requires finding amanifold 706 embedded in this joined space that represents these datapoints. A non-limiting example to compute such a manifold is a Mixturesof Probabilistic Principal Component Analyzers (Tipping. M. E. andBishop. C. M., Neural Computation, 11, 443-482, 1999). Given thismanifold 706, a motor command is computed from a given sensory input asfollows. Points 708 on the manifold are identified that have a sensorycomponent that matches the given sensory input 707. One of these Pointsis selected and its control-command component 709 is evaluated. Anon-limiting example to compute such points on the manifold and selectone of them is described in (Heiko Hoffmann. Wolfram Schenck. RalfMöller. Biological Cybernetics. Vol. 93, pp. 119-130, 2005). Thecomponent 709 is the desired control command (depicted as element 510 inFIG. 5).

Based on the learned relationship (i.e., the sensory-motor mapping)between the tactile sensory data (sensory state) and the controlcommand, the computer converts a sensory feedback signal into controlcommands 510 to achieve a desired task. In other words, the system canthen generate a series of control commands 510 in a closed-loop based onthe sensory-motor mapping to manipulate the tool against the surface,e.g., to keep the tool in contact with the surface at constant force.

In the actual control task, only the series of control commands 510 areneeded. The previous operations are carried out in preparation afterfirmly holding the tool. Whenever, the tool moves in the robot gripper,the preparatory steps (i.e., 500, 502, 504, and 506) need to be repeatedbefore using series of control commands 510 again.

An aspect of this process is to autonomously learn the sensory effect ofa robotic action and to learn the link between a lower-dimensionalrepresentation of the tactile input and the corresponding controlcommands. The control commands result in tactile input that variespredominantly in the relevant dimensions for control. Thus,dimensionality reduction extracts the relevant information for control.

As can be appreciated by one skilled in the art, the present inventionis not limited to tool manipulation but can be extended to other tasks,non-limiting examples of which include inserting a key and slidingobjects. Further, the online tactile-exploration strategy of the presentinvention can be applied to higher-dimensional motor commends (e.g.,two-dimensional instead of one-dimensional).

Moreover, the concept of learning online can be extended to theuncertain interface between the robot gripper and tool without learningthe full robot kinematics and dynamics. For example, other control andsensor values, such as force control and image features of the tool, canbe implemented. As a non-limiting example and as depicted in FIG. 8, animage sensor 800 (e.g., camera) can be included for capturing an imageof the tool 402 as connected with the robotic gripper 300. In thisexample, although the tactile sensors 302 are optional, they can be usedto further enhance the control commands in the feedback loop. The imagesensor 800 is used to capture an image of the tool 402, which is thenused to identify a desired point 802 on the tool 402 for moving thepoint 802 into contact with a surface. In order to identify the desiredpoint 802, the image is sent to the computer 304 (through the sensorinterface 306) which includes a tool recognition/feature extractionmodule. The tool recognition/feature extraction module uses any suitabletechnique that is operable for identifying an object and/or extractingobject features to identify a desired portion of the tool. Anon-limiting example of such a suitable technique are so-called SUFTfeatures, as described by Herbert Bay, Andreas Ess, Tinne Tuytelaars,and Luc Van Gool, in “SURF: Speeded Up Robust Features”, Computer Visionand Image Understanding (CVIU), Vol. 110, No. 3, pp. 346-359, 2008.

The feature extraction module is operable for identifying the tool 402and its various features to isolate the desired point 802 (e.g., tooltip). Thereafter, the computer 304 generates a series of controlcommands (which are passed through the control circuit 308) to cause therobotic gripper 300 to manipulate the tool 402, in a closed-loop, untilthe desired point 802 is positioned against the surface.

To confirm the functionality and advantages of the present invention, itwas tested using a simple robotic gripper as depicted in FIGS. 4A and4B. FIG. 4A is a front-view illustration of the gripper 400 and tool 402(i.e., pencil), while FIG. 4B is a side-view illustration of the gripper400 and tool 402. The test demonstrated robotic writing with a pencil ona surface with an unknown slope (i.e., not known to the robotic system).A R17 robot arm (from ST Robotics, located 103 Carnegie Center, Suite300, Princeton, N.J. 08540) was equipped with a plurality of tactilesensors, comprising two 2×2 tactile sensor 404 arrays (from InterlinkElectronics, located at 546 Flynn Road, Camarillo, Calif., 93012) at thegripper 400.

One tactile sensor array was placed on each gripper 400 finger. A pencilwas placed arbitrarily between the tactile sensors 404. Thus, it was notknown a priori which sensors would be in contact with the tool 402.Prior to drawing, the robot explored online the relationship betweentactile response and gripper movements. On a sinusoidal curve, thegripper 400 moved up and down touching the surface with the pencil. Thegripper 400 moved for three periods. All eight sensor values at 60 timesteps (uniformly distributed along the three periods) were collected. Onthe resulting tactile sensory data, a principal components analysis wascomputed and the direction of maximal variance (first principalcomponent) was extracted. Then, all sensory values were projected ontothis component. The resulting relationship between the projected sensorvalues and corresponding height of the gripper is depicted in FIG. 9.

FIG. 9 is a graph 900 illustrating the relationship between roboticgripper location (Z-axis displacement 902) and low-dimensionalrepresentation of tactile input (sensor value projections 904). Recordedvalues from one exploration trial are shown together with a linear fitto the data (solid line 906).

To learn this relationship and without implying a limitation, linearregression (ordinary least squares) was used between the gripper heightand sensor representation (sensory input projected onto first principalcomponent). The desired sensor representation was set to the mean ofobserved values during exploration. During drawing, the robot grippermoved uniformly in the horizontal direction, with the height of thegripper being controlled. The robot could draw on a surface with unknownslope based on tactile feedback despite uncertainty of the pencil-sensorinterface (the average deviation of the pen tip from the slope was 0.8millimeters (mm)). The pencil drawing was tested for several trialsvarying the orientation of the pencil in the gripper and the slope ofthe drawing surface. The results were consistent across trials,including trials in which the slope changed direction.

(4) Synopsis

The present invention is directed to a system and method for robotictool manipulation. This system enables a robot through onlineexploration to adjust the tool's position given tactile sensory input.In operation, the system learns a low-dimensional representation of thesensory input (i.e., tactile sensory data) and learns the mapping fromthis representation to control-relevant position commands (i.e., controlcommands) for the robotic gripper.

Using this method, it was demonstrated that the robot could accuratelydraw (0.8 mm position error) with a pencil on a paper of unknown andvarying slope. Through online learning, the robot could automaticallyadapt to the tool (e.g., pencil) position inside the gripper.

In addition, the system can be extended to different kinds of motorcommands and sensory values so that it can be used with more tasksinvolving hand-held tools. The general process is also open tonon-linear mappings. This capability and the demonstrated flexibilityallows for autonomous tool use where a robot grasps a tool and, thus,cannot accurately pre-compute the tool's position within the robotgripper.

What is claimed is:
 1. A robotic control device for manipulating a handheld tool, comprising: a gripper, the gripper being mobile in at leastone degree of freedom; a plurality of tactile sensors attached with thegripper, each sensor operable for generating tactile sensory data upongrasping a tool based on the interface between the tool and thecorresponding tactile sensor, a computer communicatively connected withboth the gripper and the tactile sensors, the computer having a memoryand a data processor, the memory encoded with instructions, that whenexecuted, cause the data processor to perform operations of: causing thegripper to grasp the tool and move the tool into contact with a surface;via a control command, causing the gripper to perform a movement withthe tool against the surface to generate tactile sensory data;performing a dimensionality reduction of the tactile sensory data togenerate a low-dimensional representation of the tactile sensory data;learning a relationship between the low-dimensional representation ofthe tactile sensory data and the control command to generate asensory-motor mapping, by performing operations of: collecting datapoints that include corresponding pairs of sensory input signal data andcontrol commands to form a joint sensory-motor space; identifying alower dimensional manifold embedded in the joint sensory-motor spacethat represents the data points; identifying points on the lowerdimensional manifold that have a sensory component that matches a givensensory input signal; projecting the identified points onto a motorsub-space to identify control commands as reflected in the sensory motormapping; and generating a series of control commands in a closed-loopbased on the sensory-motor mapping to manipulate the tool against thesurface.
 2. The robotic control device as set forth in claim 1, whereinthe gripper is a robotic hand.
 3. The robotic control device as setforth in claim 2, further comprising an image sensor for capturing animage of the tool, and wherein the computer is further configured tocause the processor to perform operations of: capturing an image of thetool; identifying a desired point on the tool for moving into contactwith the surface; and generating a series of control commands in aclosed-loop to cause the gripper to manipulate the tool until thedesired point is moved into contact with the surface.
 4. The roboticcontrol device as set forth in claim 3, wherein the sensory dataincludes a magnitude of a force input.
 5. The robotic control device asset forth in claim 3, wherein the sensory data includes both a magnitudeand direction of a force input.
 6. The system as set forth in claim 3,wherein in identifying a desired point on the tool for moving intocontact with the surface, the system includes a tool recognition/featureextraction module that identifies the tool and extracts features of thetool to identify the desired point of the tool.
 7. The system as setforth in claim 6, wherein the desired point of the tool is the tool tip.8. The robotic control device as set forth in claim 1, furthercomprising an image sensor for capturing an image of the tool, andwherein the computer is further configured to cause the processor toperform operations of: capturing an image of the tool; identifying adesired point on the tool for moving into contact with the surface; andgenerating a series of control commands in a closed-loop to cause thegripper to manipulate the tool until the desired point is moved intocontact with the surface.
 9. The robotic control device as set forth inclaim 1, wherein the sensory data includes a magnitude of a force input.10. The robotic control device as set forth in claim 1, wherein thesensory data includes both a magnitude and direction of a force input.11. The system as set forth in claim 1, wherein in causing the gripperto perform a movement with the tool against the surface to generatetactile sensory data, the movement includes movements selected from agroup consisting of random movements, zig-zag movements, rectangularoscillation, periodic movements, and pseudo-random movements.
 12. Acomputer program product for manipulating a hand held tool, the computerprogram product comprising computer-readable instructions stored on anon-transitory computer-readable medium that are executable by acomputer having a processor for causing the processor to performoperations of: causing a gripper, having tactile sensors, to grasp atool and move the tool into contact with a surface; via a controlcommand, causing the gripper to perform a movement with the tool againstthe surface to generate tactile sensory data from the tactile sensors;performing a dimensionality reduction of the tactile sensory data togenerate a low-dimensional representation of the tactile sensory data;learning a relationship between the low-dimensional representation ofthe tactile sensory data and the control command to generate asensory-motor mapping, by performing operations of: collecting datapoints that include corresponding pairs of sensory input signal data andcontrol commands to form a joint sensory-motor space; identifying alower dimensional manifold embedded in the joint sensory-motor spacethat represents the data points; identifying points on the lowerdimensional manifold that have a sensory component that matches a givensensory input signal; projecting the identified points onto a motorsub-space to identify control commands as reflected in the sensory motormapping; and generating a series of control commands in a closed-loopbased on the sensory-motor mapping to manipulate the tool against thesurface.
 13. The computer program product as set forth in claim 12,further comprising instructions for causing the processor to performoperations of: capturing an image of the tool using an image sensor;identifying a desired point on the tool for moving into contact with thesurface; and generating a series of control commands in a closed-loop tocause the gripper to manipulate the tool until the desired point ismoved into contact with the surface.
 14. The computer program product asset forth in claim 13, wherein in causing the gripper to perform amovement with the tool against the surface to generate tactile sensorydata from the tactile sensors, the sensory data includes a magnitude ofa force input.
 15. The computer program product as set forth in claim12, wherein in causing the gripper to perform a movement with the toolagainst the surface to generate tactile sensory data from the tactilesensors, the sensory data includes a magnitude of a force input.
 16. Acomputer implemented method for manipulating a hand held tool, themethod comprising an act of causing a computer having a processor andnon-transitory machine readable memory to execute instructionsspecifically encoded in the memory, such that upon execution, theprocessor performs operations of: causing a gripper, having tactilesensors, to grasp a tool and move the tool into contact with a surface;via a control command, causing the gripper to perform a movement withthe tool against the surface to generate tactile sensory data from thetactile sensors; performing a dimensionality reduction of the tactilesensory data to generate a low-dimensional representation of the tactilesensory data; learning a relationship between the low-dimensionalrepresentation of the tactile sensory data and the control command togenerate a sensory-motor mapping, by performing operations of:collecting data points that include corresponding pairs of sensory inputsignal data and control commands to form a joint sensory-motor space;identifying a lower dimensional manifold embedded in the jointsensory-motor space that represents the data points; identifying pointson the lower dimensional manifold that have a sensory component thatmatches a given sensory input signal; projecting the identified pointsonto a motor sub-space to identify control commands as reflected in thesensory motor mapping; and generating a series of control commands in aclosed-loop based on the sensory-motor mapping to manipulate the toolagainst the surface.
 17. The method as set forth in claim 16, furthercomprising an act of causing the processor to perform operations of:capturing an image of the tool using an image sensor; identifying adesired point on the tool for moving into contact with the surface; andgenerating a series of control commands in a closed-loop to cause thegripper to manipulate the tool until the desired point is moved intocontact with the surface.
 18. The method as set forth in claim 17,wherein in causing the gripper to perform a movement with the toolagainst the surface to generate tactile sensory data from the tactilesensors, the sensory data includes a magnitude of a force input.
 19. Themethod as set forth in claim 16, wherein in causing the gripper toperform a movement with the tool against the surface to generate tactilesensory data from the tactile sensors, the sensory data includes both amagnitude and direction of a force input.